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I. Problem: Active Learning without Standards for Intellectual Quality 

A variety of school reform proposals advocate moving from traditional teacher-centered 
teaching toward more progressive, student-centered or constructivist classrooms. 1 Important 
distinctions might be made among these proposals, but they reflect a common interest in students 
actively constructing meaning, grounded in students' experience, rather than students simply 
absorbing and reproducing knowledge transmitted from subject-matter fields. This common 
dimension might be summarized as active learning. 

A host of activities have the potential to engage students in thinking, problem solving, 
and construction of meaning. These include small group discussion; cooperative learning tasks; 
independent research projects; use of hands-on manipulatives, scientific equipment and arts/crafts 
materials; use of computer and video technology; community-based projects such as surveys or 
oral histories; and service learning. 

Students often show more animated, visible engagement in these activities, and teachers 
may interpret heightened engagement as student learning. But even highly active students can 
produce work that is intellectually shallow and weak. We have observed situations like the 
following: students working diligently in small groups to complete routine mathematics or 
vocabulary assignments, but one student gives the answer for others to copy; students completing 
interviews of community residents, with all questions prespecified by the teacher and the 
students' merely recording respondents' short answers, without trying to interpret their 
cumulative meaning; students using the card catalogue, computers, and mathematics 



'For diverse examples, see Bruer (1993), Brooks & Brooks (1993), Wells & Chang-Wells 
(1992), Elmore (1990), Marshall (1992), Sizer (1992). 
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manipulatives to reinforce superficial exposure to fragments of knowledge without promoting 
in-depth understanding of an idea. 

Reform efforts focused on active learning may lead down an illusory path where student 
participation in activities can become an end in itself, regardless of the intellectual quality of 
students' work. 2 All persons, regardless of their formal education "construct meaning." 
Presumably, the point of education is to improve the quality of the meanings we construct, or 
to help students "use their minds well" (Sizer, 1984). This requires standards for intellectual 
quality; that is, criteria that help to define the difference between successful, powerful, and 
laudable uses of the mind versus uses of the mind that signify mediocrity, failure or 
unproductive cognitive work. Resistance to student-centered teaching may be due in part to 
teachers and parents who have already sensed this problem. To address the skepticism, 
educators will need to show both that new approaches to pedagogy are grounded in high 
intellectual standards and that practice faithful to the standards actually enhances student 
performance. 

This article addresses each concern. First, we offer a conception of authentic pedagogy 
consistent with an active learning perspective, but which posits standards of intellectual quality, 
rather than teaching techniques or processes, as the central target of innovation. Then we 
provide empirical evidence that authentic pedagogy (defined by these standards) pays off in 
student performance across different grades and subjects. The standards of intellectual quality 
for authentic pedagogy and evidence of a link between authentic pedagogy and student 



2 Regnier (1993) offers a succinct explanation of how preoccupation with technique in 
education stifles intellectual life in schools. 



performance should advance research and practice on student-centered, or constructivist 
teaching. 

n. Authentic Human Achievement: 
A Source of Standards for Pedagogy and Student Performance 

Calls for education reform arise not simply from high drop-out rates or low test scores, 
but also from concerns that the kind of mastery required for students to earn school credits, 
grades, and high scores is frequently trivial, contrived, or meaningless. In contrast "authentic" 
academic achievement stands for accomplishment that is significant, worthwhile, and meaningful. 
Consider the kinds of mastery demonstrated by successful adults — scientists, musicians, business 
entrepreneurs, politicians, craftspeople, attorneys, novelists, nurses, designers. What key 
characteristics of their work justify calling their accomplishments authentic? And how do these 
characteristics of "real" accomplishment differ from the work that students complete in school? 3 
We define authentic academic achievement through three criteria: construction of knowledge, 
disciplined inquiry and value beyond school. 4 
Construction of Knowledge 

Persons in the diverse fields named above face the primary challenge of constructing or 
producing, rather than reproducing, meaning or knowledge. They express this knowledge in 
written and oral discourse (words and symbols in documents and conversation or speeches), by 

^his complicated matter involves controversial educational values, it has not received 
extensive scholarly study, and it beckons for interdisciplinary analysis of the relationship 
between formal attempts to educate and the ways that human competence is expressed in non- 
educational settings. The conception proposed here is based upon the work of Archbald & 
Newmann (1988), Berlak et al. (1992), Resnick (1987), and Wiggins (1993). 

4 These ideas have evolved from Archbald and Newmann (1988). The definitions that follow 
are more fully developed in Newmann, Secada & Wehlage (forthcoming). 
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making and repairing things (products such furniture, buildings, videos, sculpture), and in 
performances for audiences (musical, dramatic, athletic). In contrast, the conventional 
curriculum asks students only to identify the discourse, things, and performances that others 
have produced and codified, for example by recognizing the difference between verbs and nouns, 
between socialism and capitalism; by matching authors with their works; by correctly labeling 
rocks and body parts. 
Disciplined Inquiry 

A second defining feature of authentic academic achievement is its reliance upon a 
particular type of cognitive work: disciplined inquiry. Disciplined inquiry consists of three main 
features: use of a prior knowledge base from one or more fields; striving for in-depth 
understanding rather than superficial awareness; and expressing conclusions through elaborated 
communication. 5 

Prior knowledge base . Authentic accomplishments build on prior knowledge that has 
been accumulated in a field. The knowledge base includes facts, vocabularies, concepts, 



5 A broad definition of authentic human accomplishment might not always illustrate 
disciplined inquiry as suggested by academic study (Gardner, 1983; 1993). For example, feats 
of wilderness survival that depend largely on ingenuity and courage, forms of athletic prowess, 
or selfless acts of caring, devotion, and personal sacrifice might all be considered authentic; but 
they may not illustrate much disciplined inquiry. Since schooling, at a minimum, should 
promote academic study, this conception of human accomplishment is admittedly limited to 
achievements that depend upon the use of formal knowledge. Formal knowledge itself, of 
course, encompasses an enormous diversity in the liberal arts, applied professions and crafts, 
along with fields of literature, discourse, and practice that may not be recognized as 
H disciplines H in schools or universities. From our point of view, a field of expertise that has 
accumulated a formal knowledge base and that functions as a community of discourse to advance 
the knowledge, can be considered a discipline, even though it may not have been institutionally 
established (e.g. through awarding of advanced degrees). Examples might include stamp 
collecting, model railroads, specialized computer user groups, or sky diving. 

4 
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theories, algorithms and conventions for the conduct and expression of inquiry itself. Most 
cognitive work of school consists in transmitting prior knowledge to students. 

In-depth understanding . Disciplined inquiry tries to develop in-depth understanding of 
a problem, rather than only passing familiarity with or exposure to pieces of knowledge. Prior 
knowledge is mastered, therefore, not primarily to become literate about a broad survey of 
topics, but to facilitate complex understanding on discrete problems. In-depth understanding 
requires more than knowing lots of details about a topic. Understanding occurs as one looks for, 
tests, and creates relationships among pieces of knowledge that can illuminate a given problem 
or issue. In contrast, many of the cognitive tasks of school ask students to show only superficial 
awareness of a vast number of topics. 

Elaborated Communication . Scientists, jurists, artists, journalists, designers, engineers, 
and other accomplished adults working within disciplines rely upon complex forms of 
communication both to conduct their work and to express their conclusions. The language they 
use — verbal, symbolic, and visual — includes qualifications, nuances, elaborations, details, and 
analogs woven into extended expositions, narratives, explanations, justifications, and dialogue. 
In contrast, much of the communication demanded in school asks only for brief phrases: true 
or false, choosing from multiple choices, filling in blanks, or short sentences (e.g., "Prices 
increase when demand exceeds supply.") 
Value Beyond School 

The third distinction between authentic human achievement and conventional school 
achievement is that authentic achievements have aesthetic, rtilitarian, or personal value apart 
from documenting the competence of the learner. When adults write letters, news articles, 
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insurance claims, poems; when they speak a foreign language; when they develop blueprints; 
when they create a painting, a piece of music, or build a stereo cabinet, they try to communicate 
ideas, to produce a product or to have impact on others beyond the simple demonstration that 
they are competent. Achievements of this sort have special value which is missing in tasks 
contrived only for the purpose of assessing knowledge (such as spelling quizzes, laboratory 
exercises, or typical essay exams). The cry for "relevant," "student-centered" curriculum, is, 
in many cases, simply a less precise expression of this desire that student accomplishment should 
have value beyond being an indicator of success in school. 
Connections to Constructivism 

The perspective known as constructivism itself embraces different points of view, but 
these share a general understanding of how students learn. This understanding challenges prior 
renditions of the learning process as transmission of information to passive receivers. We 
summarize this conception of learning, its implications for teaching, and we then indicate how 
constructivist perspectives are related to the three criteria for authentic academic achievement. 6 
Learning . Learning, even of apparently quite simple material, such as the definition or 
spelling of a word, is a complex, active mental process. It is not achieved merely by 
transmitting information to a student who reproduces what has been transmitted. Instead, the 
student works on, processes, interprets, and negotiates the meaning of the information 
encountered. The student's understanding or expression of the information "taught" is 

*This synthesis of the principles of constructivist learning and teaching relies on Becker and 
Varelas (1995), Brooks and Brooks (1993), Bruer (1993), Cohen, McLaughlin, and Talbert 
(1993), Driver (1995), Marshall (1992), Newmann (1992), Nystrand and Gamoran (1991), 
Resnick (1989), Resnick, Levine, and Teasley (1991), Wells and Chang-Wells (1992), Wood 
et al. (1995). 



profoundly influenced by several factors: the student's prior knowledge, which itself is often 
far more substantial than previously assumed; the social context of values, expectations, rewards 
and sanctions in which the information is initially communicated and later expressed by the 
student; and the student's self-monitoring the process of learning. In short, students are 
constantly working to "make sense" of what they encounter, but how they do it depends much 
upon their own prior experience and the nature of social interaction that surrounds presentation 
of information and its later expression by the student. 7 

Teaching . As a descriptive theory of learning, constructivism may imply no logically 
direct prescriptions for how teachers should help children to learn. But the literature suggests 
several principles for practice that would seem to facilitate the learning process just described. 

First, teachers must be familiar with, respect and actively use students' prior knowledge 
as they teach* Students' assimilation of new information will depend upon the extent to which 
the information helps them explain or meaningfully extend their known experience. Second, 
realizing that students are complex thinkers trying to make sense of the world, teachers must 
emphasize opportunities for higher order thinking and in-depth understanding, rather than only 
rote learning and superficial coverage of information. Third, instruction must offer multiple 
opportunities for students to use conversation, writing, and other forms of expression to process 
information. Without substantial efforts in extended expression, students' efforts to make and 
negotiate meaning will be stifled. Fourth, rather than taking the role of authoritative dispenser 
of information and truth, the teacher becomes a coach, facilitator, guide, a mentor in a 

7 Von Glaserfeld (1989) explains that constructivism is rooted in the epistemological principle 
that knowledge consists not in the discovery of a separate objective ontological reality, but in 
adaptive cognition which individuals use to organize their unique experiential worlds. 
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"cognitive apprenticeship," who inspires and nudges the student to do the active work of 
learning. Finally, if students are to devote the effort required for understanding, participants 
in the social setting for learning, students and teachers alike, must exemplify norms of 
collaboration, trust, and high expectations for intellectual accomplishment. 

Links to authentic academic achievement . Our emphasis on construction of knowledge 
is consistent with the constructivist perspective of the student as a meaning-making person 
continuously negotiating prior experience with new information, even within the most traditional 
classrooms. But our vision extends this descriptive claim. We stipulate that authentic 
achievement requires "construction" to reach beyond retrieval and imitation of knowledge 
previously produced by self or others. Authentic construction of knowledge involves 
application, manipulation, interpretation or analysis of prior knowledge to solve a problem that 
cannot be solved simply by routine retrieval or reproduction. 

Our concern for disciplined inquiry is less evident in the constructivist perspective, 
because that perspective generally does not prescribe better ways of constructing meaning, or 
suggest that some kinds of meanings might be more powerful or adequate than others. 
Richardson (1994) explained that constructivist principles fail to offer guidance on the role of 
formal knowledge in teaching, or when students should arrive at reasonably common, rather than 
idiosyncratic meanings. In contrast, we see disciplined inquiry as an essential source of 
intellectual quality. For academic achievement to be authentic, the meanings that students 
construct cannot be completely idiosyncratic; they must make sense to or be verified by a larger 
public. While disciplined inquiry offers no foolproof route to "truth" or to public acceptance, 
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it does offer standards that help to establish some understandings as intellectually more worthy 
than others. 

Our criterion of value beyond school is consistent with the constructivist position that 
learning will be more adaptive or powerful when students can connect new information to their 
own experiences. Students construct meaning largely in response to rewards and sanctions in 
the sociocultural context (Resnick et al., 1991, Steffe & Gale, 1995). When students' 
achievements are valued only because they contribute to a record of success in school, success 
in these tasks often carries no adaptive value, because large numbers of students consider school 
to be only a restricted, even insignificant, arena of personal experience. In educational 
philosophy, Dewey (1916) has offered perhaps the most articulate case for aiming education 
toward achievement that has value beyond school. But this point has also been well supported 
through recent research on the nature of learning (Resnick, 1987). 

We may find support for these criteria in prior literature, but we believe this formulation 
is necessary to sharpen two critical distinctions. First, techniques or procedures such as having 
students talk more or promoting independent research, offer no assurance that students will 
produce work of high intellectual quality. As explained in the beginning of this article, any 
technique, whether student-centered or otherwise, must be aimed toward standards of intellectual 
quality. Our three criteria define intellectual standards essential for authenticity, but they do 
not offer a comprehensive set of standards for intellectual quality or for grading students. For 
example, judgments about the quality of pedagogy and student learning will also depend on at 
least on two other standards for intellectual quality: the accuracy of what is taught and learned 
and the appropriateness of what is taught and learned. Accuracy refers to the extent to which 
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the material reflects the best authoritative expertise for answering the questions asked. 
Appropriateness refers to the extent to which the material is considered significant and important 
by those with authority to exercise some control over curriculum (e.g., government officials, 
parents, professional associations). Criteria for grading may also include student cooperative 
behavior, following instructions, or completing assigned work, regardless of quality. 
Articulating more specific standards for accuracy and appropriateness of content or for student 
grading was beyond the scope of this study. Bruer's (1993) careful examination of learners' 
cognitive activity seems consistent with this point. His emphasis upon general intellectual skills, 
metacognition, domain specific knowledge, and transfer call primary attention to the intellectual 
quality of classroom work rather than to techniques. His summary of insights from cognitive 
science clearly supports our concern for construction of knowledge and disciplined inquiry, with 
at east implicit support for "value beyond school." Second, to maximize the probability that 
students' school achievements have adaptive benefit, they must have some value beyond' 
certifying success in school. But this criterion alone is not sufficient for intellectual quality. 
Construction of meaning through disciplined inquiry is what makes some solutions to real life 
problems more authentic than others. 
Implications for School Vision and Pedagogy 

All three are necessary, but not sufficient . Construction of knowledge, through 
disciplined inquiry, to produce discourse, products or performance that have value beyond 
success in school, can serve as standards of intellectual quality for assessing the authenticity of 
student performance. All three criteria are important. Students might confront a complex 
calculus problem demanding complex construction of knowledge and disciplined inquiry, and 
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solving the problem may be interesting. But if the solution has no value beyond proving 
competence to pass a mathematics course, its authenticity is diminished. Or, a student may 
write a letter to the editor, saying she opposes a newly proposed welfare plan. This may meet 
the criteria of constructing knowledge to produce discourse with value beyond school, but if it 
shows significant errors, or only shallow understanding of the issues, it would be less authentic 
because of shortcomings in disciplined inquiry. 

The conception of authentic academic achievement is demanding in its insistence on all 
three standards. The ideal is to strive for all three, but one would not expect all instruction and 
assessment activities to meet all three standards all of the time. For example, repetitive practice, 
retrieving straightforward information, and memory drills may be necessary to build knowledge 
and skills necessary for authentic performance, or to prepare for inauthentic tests required for 
advancement in the current educational system. The point is not to abandon all traditional forms 
of school work, but to keep authentic achievement clearly in view as the valued end. 

The three criteria serve as standards of quality that transcend specific practices or 
activities. That is, any particular lecture, classroom discussion, cooperative group activity, 
computer assignment, set of portfolio entries, essay question, student's oral presentation, video 
or graphic design could be evaluated on the extent to which it promoted or manifested 
construction of meaning, disciplined inquiry, and value beyond success in school. Since the 
standards can be applied across grade levels and subject areas, they may serve as a focus for 
school-wide consensus on educational mission. 

Specific standards for pedagogy and student performance . How might the three general 
criteria for authentic academic achievement be translated into standards to guide classroom 
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practice? To ascertain the degree of authentic pedagogy and student performance in 24 
restructured schools, staff of the School Restructuring Study (SRS, described below) constructed 
more specific standards to assess the level of authentic academic work observed in daily lessons, 
in assessment tasks that teachers used to evaluate student performance, and in students 1 written 
responses to the tasks. We considered pedagogy to be a combination of teachers' daily 
instruction and their assessment tasks. Student performance was indicated by their responses to 
the assessment tasks. The standards for pedagogy and performance are summarized in Table 

INSERT TABLE 1 ABOUT HERE 

HI. Authentic Pedagogy and Student Performance in Restructured Schools: 

The Empirical Study 

Having formulated standards for authentic intellectual quality that can be applied to both 

pedagogy and student performance, we now examine the actual distribution of authentic 

pedagogy and its connection to student performance in a set of restructured schools. 

Central Questions 

We focus on three central issues: the quality of and variability in observed authentic 
pedagogy and student performance, the link between pedagogy and performance, and the 
equitable distribution of authentic pedagogy and authentic student performance. 



*For both instruction and assessment tasks, specific standards were constructed for each of 
the three general standards. However, for student performance, a standard for value beyond 
school was not developed. The reason for this omission is that within the SRS, it was not 
possible to collect valid information on the actual meaning or value of each student's 
performance to the student or to an audience beyond school. Judgments of this sort would have 
required interview or survey data from students that was not obtainable for logistical reasons. 

12 
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1. Quality and variability . To what extent do teachers in restructured schools offer 
authentic pedagogy according to these standards, and how much variation is there between 
teachers, schools, grade levels, and subjects? As explained below, the SRS searched for and 
studied elementary, middle and high schools nationwide that had made significant progress in 
school restructuring and that seemed committed to aspects of authentic instruction. Other 
research has indicated great difficulties in the implementation of teaching consistent with some 
of these standards (e.g., Cohen, 1990; Peterson, McCarthey, & Elmore, in press), but the SRS 
offered the first opportunity to examine the question across all three grade levels and two 
different subjects (mathematics and social studies). 

2. Links to student achievement . To what extent does authentic pedagogy contribute to 
authentic student performance? The case for the benefits of constructivist or authentic teaching 
has relied more on analytic and philosophical arguments than extensive empirical evidence, and 
so answering this question is critical. The relationship of constructivist teaching to student 
achievement has been studied in disparate contexts, but primarily in lower elementary school in 
mathematics and language arts. Some of these studies have used both conventional and more 
authentic measures of achievement (e.g., Carpenter, Fennema, Peterson, Chiang, & Loef, 1989; 
Tharp, 1982). But we know of no study that has explored the connection of authentic pedagogy 
to achievement through an integrated set of indicators for instruction, assessment and student 
performance across the three grade levels. 

3. Equity . A major goal for reform is to enhance educational opportunity for students 
from educationally disadvantaged backgrounds. To explore this, we ask (a) to what extent are 
students from certain social and academic backgrounds more likely to receive authentic 

13 



pedagogy? (b) to what extent does authentic pedagogy bring different achievement benefits to 
students of different social and academic backgrounds? and (c) regardless of the effects of 
authentic pedagogy, to what extent is student authentic academic performance influenced by their 
social and academic background? 

Methodology 

The School Restructuring Study studied 24 restructured schools intensively, each for one 
year, to understand how organizational features of schools can contribute to six valued outcomes: 
authentic pedagogy and authentic academic performance; equity for students; empowerment of 
teachers, parents, and principals; sense of community among staff and students; reflective 
professional dialogue; and accountability. This section summarizes the methodology and 
findings relevant to relationships between authentic pedagogy and authentic academic 
performance. 9 
Sampling 

Schools * Twenty four public schools, equally divided among elementary, middle and 
high schools, that had demonstrated substantial departures from typical organizational features 
were selected through a national search. Innovations common to many of these schools were 
school-based governance councils, teachers working in teams with common plaining time, 
heterogeneous ability grouping for students, instructional periods lasting from one to two hours, 
extensive use of small groups in instruction, and special programs to address students' 
social/emotional needs. The schools included those that had begun anew and had hired new staff 



9 More detailed documentation dealing with variable construction, inter-rater reliability, and 
statistical results not reported here can be found in Marks, Newmann, and Gamoran (1995). 
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to fit a particular mission, as well as long-established schools trying to restructure around a 
newly defined mission. The schools were located in 16 states and 22 districts. Due to 
incomplete data from one middle school, this report includes only 23 schools. 10 The number 
of classrooms and students varies somewhat for different analyses, because, depending on the 
analysis, we limited the sample to classrooms with full data on authentic pedagogy and to 
students with full data on authentic academic performance, social background, and academic 
background. Measurement of these variables is explained below. 

Subjects, grade levels, and classes . Mathematics and social studies were studied in 
grades 4-5 for elementary schools, 7-8 for middle schools, and 9-10 for high schools. Within 
these grades, three classes each for mathematics and social studies were selected. For each 
subject, at least one of the three teachers was to have been clearly involved in the school's 
innovative efforts, and the classes were to reflect the range of student achievement in the grade 
as a whole. Each of the classes was observed four times during one year, during two one-week 
research visits. This report includes data from 504 observed lessons. 

Assessment tasks . Each of the mathematics and social studies teachers was asked to send 
from the observed class, two examples of assessment tasks that they considered valid and 



l0 Compared to a national sample of schools of all grade levels, the 24 SRS schools generally 
enrolled more students (average 777 students per school) and higher percentages of African 
American (20.6 percent) and Hispanic students (21.7 percent). About 37 percent of the students 
were on free or reduced lunch. Comparisons within grade level indicated pronounced 
demographic differences from a national sample. For example, the SRS elementary schools 
enrolled three times more Hispanics than other public elementary schools; the SRS high schools 
enrolled about twice as many African Americans; the SRS middle schools enrolled somewhat 
lower proportions of both African Americans and Hispanics. 
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important indicators of students* proficiency and understanding of the subject. 11 One task was 
sent in the fall and one in the spring. The teacher was also asked to complete a short 
questionnaire describing the conditions under which the task was administered. This report 
includes data from 234 assessment tasks which represent about 73 percent of all the tasks we 
requested. Ninety-five percent of the teachers included in this analysis provided at least one 
assessment task; 65 percent provided at least two. 

Student performance . The mathematics and social studies teachers were also asked to 
send a complete class set of student work completed in response to the assessment tasks they 
sent. Thus, the study tried to obtain two samples of student performance for each student in each 
observed class. Each student was also asked to complete a short questionnaire describing his/her 
perceptions of the task and the work. All students in these analyses submitted at least one 
sample of performance. At least two samples of performance were received from 45 percent 
of the students. This report includes data from 2128 students and 3128 samples of student 
performance. 

Msg or Variables and Scoring Procedures 

1. Authentic pedagog y. Authentic pedagogy is a composite of two measures: instruction 
and assessment. 

Each class received an instruction score based on the sum of the 4 standards of 
instruction (each standard rated on a scale from 1-5) over 4 classroom observations. Each lesson 
was scored by a CORS researcher trained to apply the standards. About 25 percent of the 



"Teachers in six schools studied during the first phase of research were asked to send in 
three assessment tasks. 
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lessons were observed by a second CORS researcher who made independent ratings on the 
standards. The overall level of agreement between two raters is estimated as a correlation of 
.78. 

Using the standards for assessment tasks, each assessment task was scored in Madison 
by a CORS researcher and by a teacher currently teaching the subject, trained by CORS staff 
who also had experience teaching the subject. To make judgments about the teachers' demands 
and expectations for students, the raters also examined the teacher task description and teacher 
comments on samples of student performance. If the two-person team did not agree on their 
initial independent ratings, they discussed the matter until consensus was reached for each 
standard. Each class received an assessment score based on the sum of the 7 assessment 
standards (rated on scales of 1-3 or 1-4) averaged over two tasks. 

The authentic pedagogy score for a class (which could range from 11-43) was computed 
by adding the instruction score (which could range from 4-20) to the assessment score (which 
could range from 7-23). Thus, the assessment part of this composite is slightly more heavily 
weighted than the instructional part. The internal consistency of the 11 item scale was .79 
(Cronbach alpha). 

2. Authentic academic performance . Each sample of student work was scored in 
Madison according to the 3 standards for student performance by a practicing teacher in the 
subject, trained by CORS staff. Each standard was scored separately (scaled from 1-4) for each 
subject and grade level. About 37 percent of the papers were scored independently by a second 
rater, randomly assigned. The overall level of precise agreement between two raters was 54 
percent, and agreement within one point was 92 percent. Each student received a total 
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performance score based on the sum of the three standards averaged across two samples of 
performance. For some analyses, individual student scores were averaged to produce class 
means on student performance. 

3. Student academic and social background . Since the effect of pedagogy on student 
achievement often depends on factors in students' backgrounds that influence achievement, we 
included measures of students academic background and social background. 

Academic background . All students in the observed classes were asked in the Fall to 
complete tests of basic knowledge. Students in observed mathematics classes were given a test 
composed of selected items from the National Assessment of Educational Progress (NAEP) for 
the appropriate grade level. Students in observed social studies classes were given a grade level 
appropriate test composed of selected items from the NAEP reading tests, along with a short test 
of writing scored by CORS staff according to NAEP standards. The overall response rate for 
these tests was about 85 percent. 12 

Social background . Students completed surveys in which they reported their gender, 
race, and whether they were of Hispanic background. They also reported on household 
resources associated with socioeconomic status, and secondary students also reported their 
parents' level of education. These indicators were combined into a measure of socioeconomic 
status. 

12 Student scores on these NAEP tests at the elementary level were at the national average 
in mathematics, slightly above in reading; at the middle level, above the national average in both 
subjects; at the high school level, below national averages (but this could be due to the fact that 
the high school NAEP norms used were based on 12th grade and most SRS students were not 
above grade 10). Students also reported on their recent grade point averages in school. These 
correlated highly with the NAEP measure. Since the NAEP measure offered a more standardized 
indicator across schools, we used this instead of grade point average in statistical analyses. 
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Findings 

Quality and Variability in Authentic Pedagogy and Achievement 

At the time of entry into the study, all schools had demonstrated clear progress in 
organizational restructuring and had shown some evidence to CORS visitors of authentic practice 
in mathematics or social studies. Nevertheless, after collecting data systematically during one 
year in each school, we found that schools varied substantially in their success on these standards 
for authentic pedagogy. In some schools, our researchers found many examples of high quality 
authentic practice in both mathematics and social studies, but in others, very few. 

The degree of authentic pedagogy observed is indicated in Table 2. The overall means 
are lower than the midpoint of our index (27). Even the most successful teachers and schools 
scored well below the highest levels of authenticity. In short, even in restructured schools, 
pedagogy was rarely rated at the higher levels of our standards. This is consistent with other 
findings that the promotion of authentic teaching is enormously difficult. 13 

INSERT TABLE 2 ABOUT HERE 

Regardless of the overall level of success on our standards, we found tremendous 
variation within the observed range. That is, both individual teachers and schools as a whole 
varied substantially relative to one another. The most successful teacher scored 33.5 and the 
least successful 12.5. The most successful school scored 54.7 (averages for the two subjects are 
added) and the least successful 33.3. Overall, about 60 percent of the variability in classroom 
scores on authentic pedagogy was due to differences between classes within schools, and about 



l3 Ball (1990), Cohen (1990), Peterson (1990a, 1990b), McCarthey &- Peterson (1993), 
Peterson et al. (in press), Prawat (1992), Wiemers (1990), Wilson (1990). 
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40 percent of the variability was due to differences between school averages. 14 Levels of 
authentic pedagogy were similar in mathematics and social studies and across the three grade 
levels. 

What are the implications of these numbers? If our standards of intellectual quality are 
appropriate goals, then there is good news and bad news. The good news is that some teachers 
and schools have been reasonably successful, signaling hope that authentic pedagogy can be 
achieved. The bad news is that overall levels of authentic pedagogy remain low according to 
these standards, even in highly restructured schools, and that some teachers and schools have 
barely begun the journey toward authentic pedagogy. 

Similarly, for student performance, overall means in each subject (Table 2) were below 
the midpoint (7.5) of the range between the lowest (3) and highest (12) possible scores, and even 
the most successful students scored below the highest levels on our measures. Yet, student 
performance varied considerably. For example, at the high school level, the most successful 
student in mathematics scored 11, the least successful 3, and in social studies the most successful 
student scored 12, while the least successful scored 3. Most of the variability in student 
achievement occurred within classes (53 percent), but variability between classes accounted for 
36 percent, and variability between schools accounted for 11 percent of total variance in 
students* authentic academic performance. Social studies performance exceeded mathematics 
at the middle and high school levels. Across the three grade levels, mathematics performance 
remained stable. In social studies, middle school performance was significantly higher than both 



14 Estimates of variance were computed using Hierarchical Linear Modeling (ELM) (Bryk 
& Raudenbush, 1992). 
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elementary and high school, and high school performance was higher than elementary school 
performance. 15 

Links Between Authentic Pedagogy and Authentic Student Performance 

Why worry about quality and variability in authentic pedagogy unless it has consequences 
for student learning? Only if authentic pedagogy actually enhances student performance does 
the variability we found have significant implications. Teachers and schools rating high on these 
standards can be assured they're on the right track, and the less successful ones could 
legitimately be urged to direct their work toward standards like these. 

An estimate of the effect of authentic pedagogy on student performance is provided in 
Table 3, which combines the results for students in mathematics and social studies. After 
controlling for students' social background (i.e. gender, race, ethnicity, socioeconomic status) 
and academic background (NAEP achievement), the effect of authentic pedagogy is .37 
(p < .001), and the analysis, using all predictors, explains about 35 percent of the variance 
among classes in student performance. 

INSERT TABLE 3 ABOUT HERE 
We examined the link between pedagogy and performance for each subject across grade 
levels as well as separately for each subject within each grade level. Comparing the two 
subjects across grade levels, we found virtually identical coefficients for authentic pedagogy (.39 
for math, .38 for social studies). The size of the authentic pedagogy effect from elementary to 
middle to high school remained stable for social studies (.39, .44, .45). The effect of authentic 



15 Since student performance was scored according to expectations within each grade le\ % 
one would not necessarily expect the scores to increase from elementary to high school. 
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pedagogy in mathematics was high in elementary and high schools (.38 and .62 respectively), 
but lower in middle schools (.20). We have no explanation for the lower effect of pedagogy in 
mathematics in middle schools, compared to high schools, but the sample size may be too small 
to allow proper comparisons. Except for this anomaly, the effect of authentic pedagogy was 

* robust across grade levels and subjects. 

The design of the study did not allow the strongest possible test of the link between 
pedagogy and performance. Ideally, performance would be judged on a common set of authentic 
tasks that gave all students an opportunity to show high levels of performance. In this study, 
however, each student's performance was a response to the teachers' unique assessment tasks, 
and the authenticity scores of those tasks contributed to the value of the independent variable 
(authentic pedagogy was constructed as the sum of the instruction and assessment task scores). 
Students of teachers with low-scoring assessment tasks did not have the same opportunity to 
demonstrate high performance as students with teachers who assigned high scoring assessment 
tasks. As might be expected, assessment tasks of lower quality were associated with lower 
student performance and also a narrower range of student performance than tasks of high 
quality. 16 

We examined the relationship of task, instruction and performance in a way that could 
reduce the dependence of student performance on the quality of the task. With two samples of 



16 HLM regression analyses indicated that assessment task quality was the major contributor 
to the strong correlation between the composite pedagogy variable and student performance. 
Observed instructional quality alone had almost no effect in social studies and in mathematics, 
a larger, but still relatively small effect. This finding supports the maxim that "what you test 
is what you get," ~ as teachers' expectations for authentic performance increase, so will the 
quality of student performance. 
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student performance (fall and spring), two assessment tasks (fall and spring) and four classroom 
observations (two fall, two spring), we divided the sample into two patterns that disconnected 
specific assessment tasks from student performance scores; namely, fall performance, fall 
instruction, spring task; and spring performance, spring instruction, and fall task. These 
analyses showed the same general pattern as indicated in Table 3. But if the measure of 
authentic performance were based on set of common tasks for all students, we suspect the link 
between pedagogy and performance would be reduced. 

Using data from the analysis in Table 3, we computed the consequences for statistically 
"average" students (i.e. those of average socioeconomic status who scored at the mean on NAEP 
achievement) of being in a class where the quality of pedagogy was average, low (1 sd below 
the mean) or high (1 sd above the mean). For example, in a class with average pedagogy the 
average white male student would score 6.1. In the low pedagogy class, he would score 5.4, but 
in the high pedagogy class, he would score 6.8. Figure 1 shows the performance consequences 
of receiving low, medium, and high pedagogy for comparable female and non-white students. 
These increments in the raw test score may seem small relative to the absolute scale of 3 to 12 
points, but they reflect substantial improvements in these students' rankings relative to their 
peers on authentic academic performance. Regardless of race or gender, an average student 
would increase from about the 30th percentile to about the 60 percentile as a result of receiving 
low versus high authentic pedagogy. This illustrates the major contribution that authentic 
pedagogy makes to authentic academic performance. 

INSERT FIGURE 1 ABOUT HERE 
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Equitable Distribution of Pedagogy and Student Performance 

In our view, effective instruction must promote authentic student achievement not just 
on average, but in a way that does not discriminate among students on the basis of their diverse 
backgrounds. Does authentic pedagogy promote high achievement for all students? We 
addressed this question in three ways. First, we considered whether students from traditionally 
advantaged backgrounds have more access to authentic pedagogy than students from 
disadvantaged groups. Second, we examined whether advantaged students benefit more from 
authentic pedagogy than other students. Does authentic pedagogy pay off more for some 
students than others? Third, we asked whether using authentic performance as our outcome 
measure introduces new biases against students from disadvantaged groups. 

Distribution of authentic pedagogy . Is authentic pedagogy more commonly experienced 
by certain categories of students? The results in Table 4 respond to this question. In step 1, 
we show that differences within schools in gender, race, ethnicity, and socioeconomic status 
(SES) are unrelated to variation in authentic pedagogy. In step 2, we observe that students' 
NAEP scores are modestly associated with the extent of authentic pedagogy: students who start 
out with higher achievement are somewhat more likely to encounter more authentic pedagogy. 
Because authentic pedagogy builds on what students know and can do, there may be some 
tendency for teachers to use it more extensively with higher-performing students. Although the 
restructured schools had substantially reduced their uses of ability grouping, most (especially 
high schools) had not eliminated it, and this may also be reflected in the results. However, the 
findings also indicate that the restructured schools had attained one important criterion for 
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equity: students from different gender, racial, ethnic, and socioeconomic backgrounds have equal 
access to high-quality instruction. 

INSERT TABLE 4 ABOUT HERE 
Differential effects of authentic pedagogy . Even though authentic pedagogy is equally 
distributed on most criteria we examined, it could still promote inequality by raising achievement 
more for some students than others. Critics of traditional instruction frequently charge that it 
serves advantaged students better than disadvantaged students (e.g., Quality Education for 
Minorities Project, 1990). Is this also the cas'~ with authentic pedagogy? To address this 
question we first tested whether the effects of social and/or academic background on authentic 
academic achievement differed across classes. Regardless of the level of authentic pedagogy, 
we found that gender, race, ethnicity, and SES exerted the same effects on performance in all 
classes. However, the effects of NAEP achievement scores on authentic academic achievement 
did vary significantly across classes. This led us to examine whether variation in authentic 
pedagogy across classes tended to boost or depress the effect of a student's NAEP score on 
authentic academic achievement. As indicated by the interaction coefficient of .08 (p < .05) 
in Table 3, the contribution of authentic pedagogy is slightly greater for students with higher 
NAEP scores. For example, if a low-achieving student (one standard deviation below the mean) 
moved from a class that was low in authentic pedagogy (one standard deviation below the mean) 
to a class that was high in authentic pedagogy (one standard deviation above the mean), he or 
she would gain about .58 standard deviations on our outcome measure. However, a high- 
achieving student making the same transfer would gain about .90 standard deviations. Although 
the difference is noteworthy, it is worth emphasizing that the main effect of authentic pedagogy 
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(.37) is more salient than the interaction (.08). Overall, authentic pedagogy helps all students 
substantially, and it gives an extra boost to high-achieving students. 

Background effects and authentic student performance . A final concern of critics is that 
tests of authentic student performance may introduce new biases against students from 
disadvantaged backgrounds (Feinberg, 1990; Wolf, Bixby, Glenn, & Gardner, 1991). Our 
analysis in Table 3 shows that whereas Hispanics and low-SES students did not score 
significantly lower than whites or high-SES students, respectively, African Americans did score 
lower than whites, and girls scored significantly higher than boys. How large are these gaps? 
As a standard of comparison, we present in Table 5 the differences on our test of authentic 
performance alongside differences on the NAEP achievement test, a more traditional measure. 
(Both outcomes are in standard deviation units.) This comparison indicates that inequality in 
authentic performance is no greater, and quite possibly less, than inequality on a traditional 
standardized test. The advantage for females is slightly larger on the authentic assessment, but 
the difference between .20 and .16 is not statistically significant. Differences in the coefficients 
for Hispanic and SES, both of which favor the authentic assessment as being less differentiating, 
are also insignificant. Moreover the coefficient of -.23 for African Americans in authentic 
performance is significantly lower than the coefficient of -.45 on the traditional test. Hence, 
while inequalities have not been eliminated, the assessment of authentic performance does not 
seem to have exacerbated the problem. 17 

INSERT TABLE 5 ABOUT HERE 

l7 Less inequality on the measure of authentic academic performance is not simply a 
reflection of possibly lower test reliability here than on standardized tests. If that were the case, 
female-male inequality would also be reduced here which did not occur. 
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IV. Summary and Implications 

Proposals for innovative teaching consistent with a constructivist perspective emphasize 
practices to make students more active learners. However, these more participatory practices, 
such as student discussions, small group exercise, hands-on activities, or independent research 
projects can be conducted without emphasizing deep understanding or meeting high intellectual 
standards. The research reported here developed standards of intellectual quality that can be 
applied to a wide range of daily lessons, teachers' assessment tasks, and the evaluation of student 
written work. The standards represent three characteristics of authentic academic achievement 
observed in significant accomplishment by adults: (a) construction of knowledge (b) through 
disciplined inquiry (c) to produce discourse, performances or artifacts that have value beyond 
certifying success in school. 

An important empirical question is whether teaching that approximates these standards 
for intellectual quality actually improves students' authentic academic performance. A study of 
about 130 classrooms in 23 restructuring public schools, equally divided among elementary, 
middle and high schools examined the quality of pedagogy and student performance in both 
mathematics and social studies. Three main findings emerged. 

First, while some classrooms and schools were far more successful than others, the 
overall levels of authentic pedagogy observed, even in this sample of highly innovative schools, 
fell well below the highest levels on the proposed standards. This is consistent with prior 
research on the difficulty of enhancing intellectual rigor in American classrooms. The good 
news is that, in spite of considerable obstacles, some teachers and schools have made 
considerable progress toward such standards of quality. This suggests that standards of this sort 
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might be useful in helping the profession move beyond the adoption of techniques and 
procedures as the focus for innovation. 

Second, authentic pedagogy does pay off in improved authentic academic performance 
for students at all grade levels and in both mathematics and social studies. It has been difficult 
to interpret the cumulative significance of prior research on the link between authentic pedagogy 
and student achievement, because previous studies have used disparate frameworks, and the most 
systematic investigations seem to have been concentrated largely at the elementary level in 
mathematics. By using a common, integrated set of standards to examine pedagogy at the three 
grade levels and in two subjects, this study adds significant empirical knowledge on this 
question. Limitations in the design of the study may cast some doubt on the extent to which we 
have established a clear causal relationship; we have not shown that interventions which 
deliberately set out to use these standards will boost student performance. On the other hand, 
the robust relationship between authentic pedagogy and student performance suggests reasonable 
grounds for working toward more deliberate use of the standards. 

Finally, we found that it is possible to deliver authentic instruction reasonably equitably, 
and that its effect on students' academic achievement is reasonably equitable. At least in this 
sample of highly restructured schools, neither gender, race, ethnicity, nor socioeconomic status 
determined which students received high or low levels of authentic pedagogy. And neither 
gender, race, ethnicity, nor socioeconomic status affected the impact of authentic pedagogy on 
authentic academic achievement, once students' prior NAEP achievement was taken into 
account. Although females performed better than males, and whites better than African 
Americans on the measure of authentic academic performance, these disparities were no greater 
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(and some were possibly less) than those commonly observed in conventional test performance. 
Whether all schools can achieve the levels of equity observed in this special sample remains to 
be seen, but at least we have an "existence proof that authentic pedagogy can be distributed 
equitably to students from all social backgrounds, with reasonably equitable benefits. Additional 
work is necessary to eliminate the disadvantage to African Americans and to males on this 
particular measure of authentic achievement and on conventional measures as well. 

As a whole, the findings support the pursuit of authentic pedagogy to help cultivate 
authentic academic performance for students. But findings on the generally low levels (and 
considerable within-school variability) of authentic pedagogy indicate that even in restructuring 
schools, the task is enormously difficult. How to enhance authentic pedagogy is beyond the 
scope of this report, but it is being addressed by studies of governance, professional community 
and the role of external agencies in the schools of this sample. 18 

The main purposes of this study were to articulate standards of authentic intellectual 
quality that can be applied to a variety of teaching practices and to examine the extent to which 
pedagogy consistent with these standards enhances authentic student performance. Theoretically, 
almost any technique, however traditional (e.g., lecture, textbooks, multiple choice tests) or 
innovative. (e.g., student discussion, primary sources, portfolios) can be applied in ways that 
stimulate high or low levels of authentic intellectual activity. The strong empirical relationship 
between these standards of quality and authentic student performance suggests that the distinction 
between practice or technique and intellectual quality may indeed deserve attention. If quality 
is more fundamental than specific technique in promoting student achievement, teachers and 
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parents may at times resist innovations in teaching such as "active learning," because they fail 
to see how these enhance the intellectual quality of student experience. On the other hand, if 
the implementation of student-centered, or constructivist practices were guided by explicit 
standards for authentic intellectual quality, this study indicates that student performance would 
benefit. 
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Table 1 

Standards for Authentic Pedagogy and Student Academic Performance 1 
Authentic Pedagogy 

A, Classroom Instruction 

1, Higher Order Thinking : Instruction involves students in manipulating information and 
ideas by synthesizing, generalizing, explaining, hypothesizing, or arriving at conclusions that 
produce new meanings and understandings for them. 

2, Substantive Conversation : Students engage in extended conversational exchanges with 
the teacher and/or with their peers about subject matter in a way that builds an improved and 
shared understanding of ideas or topics. 

3, Deep Knowledge : Instruction addresses central ideas of a topic or discipline with 
enough thoroughness to explore connections and relationships and to produce relatively complex 
understandings. 

4, Connections to the World Beyond the Classroom : Students make connections between 
substantive knowledge and either public problems or personal experiences. 

B. Assessment Tasks 

1. Or ganization of Information : The task asks students to organize, synthesize, interpret, 
explain, or evaluate complex information in addressing a concept, problem, or issue. 

2. Consideration of Alternatives : The task asks students to consider alternative solutions, 
strategies, perspectives, or points of view as they address a concept, problem, or issue. 

3. Disciplinary Content : The task asks students to show understanding and/or use of 
ideas, theories, or perspectives considered central to an academic or professional discipline. 

4. Disciplinary Process : The task asks students to use methods of inquiry, research, or 
communication characteristic of an academic or professional discipline. 

5. Elaborated Written Communication : The task asks students to elaborate their 
understanding, explanations, or conclusions through extended writing. 



*Taken from Newmann, Secada, & Wehlage (forthcoming). 
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6. Problem Connected to the World : The task asks ludents to address a concept, 
problem, or issue that is similar to one that they have encountered or are likely to encounter in 
life beyond the classroom. 

7. Audience Beyond the School : The task asks students to communicate their knowledge, 
present a product or performance, or take some action for an audience beyond the teacher, 
classroom, and school building. 

C. Authentic Academic Performance 

1. Analysis : 

(a) Mathematical Analysis : Student performance demonstrates and explains their thinking 
with mathematical content by organizing, synthesizing, interpreting, hypothesizing, describing 
patterns, making models or simulations, constructing mathematical arguments, or inventing 
procedures. 

(b) Social Studies Analysis : Student performance demonstrates higher order thinking 
with social studies content by organizing, synthesizing, interpreting, evaluating, and 
hypothesizing to produce comparisons/contrasts, arguments, application of information to new 
contexts, and consideration of different ideas or points of view. 

2. Disciplinary Concepts : 

(a) Mathematics : Student performance demonstrates an understanding of important 
mathematical ideas that goes beyond application of algorithms by elaborating definitions, making 
connections to other mathematical concepts, or making connections to other disciplines. 

(b) Social Studies : Student performance demonstrates an understanding of ideas, 
concepts, theories, and principles from the social disciplines and civic life by using them to 
interpret and explain specific, concrete information or events. 

3. Elaborated Written Communication : 

(a) Mathematics : Student performance demonstrates a concise, logical, and well 
articulated explanation or argument that justifies mathematical work. 

(b) Social Studies : Student performance demonstrates an elaborated account that is clear, 
coherent, and provides richness in details, qualifications and argument. 
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Table 3 

The Relationship of Authentic Pedagogy to Authentic Academic Performance for 
Mathematics and Social Studies Combined* 





Dependent Variable 


Variables 


Authentic Academic Performance 


Intercept 


-.07 


Female 


.10 ** 


African American 


-.16 * 


Hispanic 




oocioccononuc ouiius 




NAEP Achievement 


.27 *** 


Class Average NAEP Achievement 


-.01 


Class Authentic Pedagogy 


37 +++ 


NAEP Achievement-by- 
Authentic Pedagogy* 


.08 ** 


Percentage of Between-Class 
Variance Explained: 




In Average Authentic 
Academic Performance 


34.7% 


In Effects of NAEP Achievement 
on Authentic Academic Performance 


28.5% 



11 Estimates from a 3 -level multilevel model computed with the HLM program (Bryk and Raudenbush, 1992). Level 
3 (school level) has no predictors. Cases with missing values are deleted from the analysis. The final analytic 
sample includes 2,110 students in 128 classrooms in 22 schools. 



b All continuous variables are standardized. Female, African American, and Hispanic are indicator variables, coded 
l=Yes, 0=No. The coefficients for the indicator variables represent the increment added to the intercept, i.e., 
average authentic academic performance, for female gender, African American race, and Hispanic ethnicity. The 
coefficients for the continuous variables represent the increment added to the intercept for a standard deviation 
increase in the independent variable. 

c Multilevel interaction of student NAEP Achievement by class authentic pedagogy. 

* p £ .05 ** p <; .01 *** p £ .001 
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Table 4 

Social Distribution of Authentic Pedagogy* 





Step 1 


Step 2 


Interceot 


- 13 


- 11 


Female 


04 


09 


African American 


-.02 


.05 


Hispanic 


.07 


.11 


SES 


-.02 


-.04 * 


NAEP 

Achievement 




.14 *** 


% Within-School 
Variance Explained 


0.0% 


2.2% 



1 Mathematics and social studies combined: A within-school HLM analysis with effects fixed. 
Cases with missing values are deleted from the analysis. The final analytic sample includes 
2,899 students in 23 schools. Authentic pedagogy is measured on 130 teachers. 
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Table 5 

Students' Social Background and Achievement: SRS and NAEP Achievement Compared* 







Dependent Variables 




SRS Achievement 


NAEP Achievement 




. XO 




Female 




U *** 

• lu 


African American 


-.23 * 


-.46 *** 


Hispanic 


-.13 


-.25 


SES 


*** 


*** 


% Within-School 
Variance Explained 


2.7% 


6.5% 



a HLM within-school analyses with effects fixed. Cases with missing values are deleted from 
the analysis. With SRS achievement as the dependent variable, the sample includes 2244 
students; with NAEP achievement as the dependent variable, the analysis includes 2899 students. 
Both analyses include 23 schools. 



* P < .05 *** P < .001 
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